Evaluation of Parallel Simulations on Multi-core Clusters of Miscible Displacement Applications

In this work, we analyze a parallel finite element implementation for multi-core clusters of incompressible miscible displacements in porous media. For that, we compare two alternatives of MPI jobs scheduling on the clusters’ cores. One of them, named fill-up, schedules jobs to all cores of a Central Processing Unit (CPU) of a cluster before considering the next CPU, and the other, named single-core, schedules only a single job for each CPU. The speedup and CPU time are calculated and compared for two multi-core clusters of the Laboratório de Computacação de Alto Desempenho (LCAD) at the Universidade Federal do Espı́rito Santo and a multi-core cluster of the Núcleo de Computação o de Alto Desempenho (NACAD) at the Universidade Federal do Rio de Janeiro. The parallelization of the finite element method uses a domain decomposition strategy with block-arrowhead structure of the resulting matrix, special sparse matrix storage formats and the Message Passing Interface (MPI) to exchange information across the sub-domains. Our results showed that, although modern machines have a large number of cores and are faster than previous ones, the competition of these cores for the CPU memory hierarchy resources has a strong impact on the performance of parallel applications.